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A study was made of the effect of various learner 
and task characteristics on learnino from scrambled frames in an 
instructional rroaran. ^he independent variable? were: sequence 
(scrambled vs. logical), grade, sequence length, criterion test 
complexity, sex, T 0, mathematics achievement, and reading 
comprehension. The dependent variables were; time in acquisition, 
errors in acquisition, immediate nosttest scores, an* retention test 
scores. The ohlective of the program was to teach how to compute line 
slopes, and the learning task was empirically shown to be linear. 
Multi-factor analyses of variance were performed on numerous 
combinations of variable?, orade level, mathematics achievement r an* 
test complexity did not interact with sequence, but reading, sex, *0# 
and sequence length did. Differences v°r° observed in errors and time 
in acquisition, but not in performance on the criterion tests. m he 
competence test? used arc discussed ir *he appendix. References are 
given. (Author/M r ) 
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Purpose of the Study 

One of the major principles of instructional programming is that 
each item must be constructed in such a way as to promote positive trans 
fer to the succeeding item, A number of studies (Gavurin and Donahue, 
1961; Roe, 1962; Roe, Case and Roe, 1962; Levin and Baker, 1963; 
Hamilton, 1964; Payne, Krathwohl and Gordon, 1967; Niedermeyer, Brown 
and Sulzen, 1969; Brown, 1970) have tested this principle by examining 
the effects of learning in programs in which the instructional frames 
have been deliberately scrambled, These studies generally concluded 
that item scrambling did not harm, and possibly even helped learning. 

Except for three studies which looked at the effects of 
sequencing under conditions of mathematics aptitude, IQ or response mode 
the studies were not designed to converge on the specific causes of the 
differences in learning when and if differences appeared. 

It was the purpose of the present study to examine the effect 
that various learner and task characteristics might have on learning in 
scrambled item sequences. 



Jfethodology 

In attempting to discover the conditions under which scrambled 
sequences affect learning, the following independent variables were 
manipulated, 

1* Sequence i logical and scrambled 
2* Grade ; 6, 7 and 8 

3, Sequence Length ; 9, 17 and 70 frames 

4, Criterion Test Complexity ; comprehension, application and analysis 
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Items . 

5* Sex : male and female 

6. I_q: low, middle and high (72 to 95, 96 to 105, 106 to 143 

respectively) 

7. Mathematics Achievement : low, middle and high (-3.4 to -.8, -.7 to 

+.7, +.8 to +4.2 grade equivalents respectively) 

8. Reading Comprehension : low, middle and high (-3.1 to -.8, -.7 to 

+.7, +.8 to +4.5 grade equivalents respectively) 

The following four dependent variables were used as criteria of 
learning. 

1. Time to complete sequences 

2. Errors made in sequences 

3. Immediate posttest scores 

4. Retention test scores (administered 14 days after the treatments) 

The instructional objective of the program was to teach learners 
in Grades 6 to 8 how to compute line slopes. A previous study (Olsen, 
1968) had empirically determined that the learning tasks of the program 
were linearly related. 

Ihe program was in booklet form, with the correct responses 
appearing on the back of each page. Linear programming techniques were 
used in constructing the basic program, and considerable attention was 
given to keeping the learners actively engaged in writing and drawing. 
Each frame was constructed as a discrete learning experience, and inter- 
frame dependency was avoided. 



4 



In order to determine the learning gains attributable to the 
basic program, the Solomon Four-Group Design was used for factoring out 
the effects of pretesting. It was determined that the program increased 
the competence of the subjects in each of Grades 6, 7 and 8 to at least 
the p < .0005 significance level. Ihe mean competence gains amounted 
to 42%, 73%, and 44% within Grades 6, 7 and 8 respectively. 

The total program was composed of three sections of different 
lengths. The item scrambling was carried out by randomly ordering the 
individual items within each of the three sections. 

The immediate posttest and retention test contained problems at 
different levels of complexity in order to determine if the subjects 
under the scrambled condition could solve problems as well as the subjects 
under the logical condition when it came to problems at different levels 
of complexity (Appendix A). 

Multi-factor analyses of variance were performed on a large 
number of combinations of the Independent and dependent variables in 
order to examine the interactive effects of the sequences and the various 
learner, task, and criterion test characteristics. 

Data Sources 

The subjects were chosen from Grades 6, 7 and 8 in two rural- 
suburban school districts. Ihe subjects* IQ scores ranged froir. 72 to 
143, with a mean of 107.6. Their mathematics achievement scores ranged 
from 3.4 grade equivalents below to 4.2 grade equivalents above their 
actual grade location, with a mean of +.6. Their reading comprehension 
scores ranged from 3.1 grade equivalents below to 4.5 grade equivalents 
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above their actual grade location, v/ith a mean of +.7. There were 327 
subjects In the total sample. 

Findings 

Regardless of the combinations of Independent variables in inter- 
action with sequence, the only meaningful significant differences that 
occurred regarding sequence were the following (Appendix B). 

1. When the subjects were working in the longest sequence in the pro- 
gram, the middle reading subjects receiving the scrambled treatment 
made significantly (p <,001) more errors in the program than the 
middle reading subjects receiving the logical treatment. 

2. When the subjects were working in the longest sequence in the pro- 
gram, the males receiving the scrambled treatment took significantly 
(p<.05) more time to advance through the program than the males 
receiving the logical treatment. Within the total group receiving 
the scrambled treatment, the males took significantly (p<.05) 

more time to complete the program than the females. 

3. When the subjects were working in the longest sequence in the pro- 
gram, the middle IQ subjects receiving the scrambled treatment took 
significantly (p<.01) more time to advance through the program than 
the middle IQ subjects receiving the logical treatment. Within the 
total group receiving the scrambled treatment, the middle and high 
IQ groups both took significantly (p <.001 in both cases) more time 
to complete the program than the low IQ group. 

Grade level, mathematics achievement and criterion test complexity 
did not significantly interact with sequence in any way, but reading 
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comprehension, sex, IQ and sequence length significantly affected the 
number of errors made or the amount of time spent in the program, 

Although the middle reading comprehension, middle IQ, and male 
subjects made more errors or took more time in completing the program 
under the scrambled condition, they scored as highly on the posttest 
and retention tests as their counterparts under the logical condition. 

Conclusions 

Mathematics was chosen as the learning content of the experiment 
because it is generally considered to possess a more definite structure 
than other school learning areas such as history, English, geography, 
and so on, It was felt that if scrambled learnings have significant 
effects on the extent to which individuals learn, then these effects 
would be likely to be found in the study of mathematical learning tasks, 
Of eight previous item sequencing studies, six used mathematical tasks, 
one used a task in psychological terminology, and one used a task in the 
recognition of music symbols. Significant differences were found only 
in the studies using mathematics sequences* 

It is thus felt that the statistically nonsignificant findings 
of the study generalize to areas of learning which possess structures of 
less rigidity than mathematics. That is, if item scrambling does not 
affect learning in mathematics, then it also will not affect learning 
in content areas of less structural rigidity* 

Likewise, the statistically significant findings of the study 
generalize to learning in content areas which have structures similar 
to that of mathematics, or which have structures of greater rigidity, if 
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such exist. 

Three characteristics that were previously studied in interaction 
with sequence were mathematics aptitude, IQ, and response mode. But for 
the most part, the previous studies were not designed to converge upon 
the causes of learning differences when and if differences appeared. 

The present study manipulated a large number of variables in an attempt 
to determine the effects of various learner and task characteristics on 
learning In scrambled item sequences. If item scrambling has seriously 
detrimental effects on learning, then this study should have identified 
the causes *)f some of them. 

The findings led to the following conclusions. 

1, Ihe linearity of the underlying structure of the learning task does 

not appear to be an important factor in affecting learning in scrambled 
item sequences, 

2* The effects of item scrambling are the same at different grade levels, 
provided that the instructional program is constructed for the grade 
level concerned. 

3, The usual predictors of mathematical performance such as IQ, 

mathematics ability, and mathematics achievement are not useful in 
predicting how subjects will perform in scrambled mathematics programs* 
4* The scrambling of instructional programs does not prevent subjects 
from learning either simple or complex skills. 

5. Although boys subsequently perform as highly as girls on posttests 
and retention tests, and possess the same mastery of the skills being 
taught, they require more time than girls to learn from scrambled 
item sequences* 
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6* Although subjects with different IQ's subsequently perform equally 
well on posttests and retention tests, and possess the same mastery 
of the skills being taught , subjects with average IQ's require more 
time than subjects with lower or higher IQ's to learn from scrambled 
item sequences . 

7. Although subjects at different reading comprehension levels subse- 
quently perform equally well on posttests and retention tests, and 
possess the same mastery of the skills being taught, subjects at 
average reading comprehension levels make more errors while learning 
from scrambled Item sequences than subjects at lower or higher read- 
ing comprehension levels. 

8. Differences in learning from scrambled item sequences are more likely 
to be detected in longer learning sequences than in shorter ones. 

(The previous studies did not vary sequence length, and one study 
used a sequence which was only 10 frames in length.) 

9. When learning sequences are scrambled, the results are more likely to 
be detrimental than helpful to learning. (Some of the previous studies 
suggested that item scrambling helped learning rather than hindered it.) 

10. Skinner's principle of careful sequencing is supported when it comes 
to time and errors in learning. 

11. Gagne's theory of learning hierarchies is supported when it comes to 
time and errors in learning. 

12. When item scrambling does produce detrimental effects to learning, 
these effects are more relative to the efficiency of instruction and 
learning than to the amount or type of learning that is involved. 

'o 
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Speculation , — In post-treatment interviews the subjects re- 
ceiving the scrambled sequence treatments commented as favorably on 
the program as the subjects receiving the logical sequence treatments, 
The scrambled sequence group did not notice that the learnings were 
scrambled , 

It appears that the sequencing studies have demonstrated one 
of the facts of life — that most of what we learn comes to us in dis- 
organized lots. This being the case, the human organism has had 
sufficient practice in naturally disorganized learning to cope with 
artificially disorganized learning. 

Implications 

The study has a number of implications for both researchers 
and practitioners, 

1, In research in sequence theoty, longer rather than shorter sequences 
should be used if significant effects are to be discovered, 

2, It may be that item sequencing has important learning effects which 
have not yet been discovered. It was found that subjects in the 
scrambled sequence groups needed more time to complete the retention 
test, although they did not need more time to complete the immediate 
post test. This may imply that long-term retention is not achieved 
as well by subjects learning in scrambled sequences, Additional 
sequence research needs to be done in the affective and psychomotor 
areas of learning as well, 

3, The findings of the study partially support the proposition that 
subjects who make errors while learning do not learn as well as 
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subjects who do not make errors while learning. Ihe subjects making 
more errors vent through the program more slowly, but they performed 
as well on the competence tests. Perhaps time in acquisition is the 
major consequence of error-making while learning. (The middle 
reading and middle IQ subjects were largely the same persons* They 
made iaore errors and needed more time in completing the scrambled 
program, but they performed as well as the logical treatment subjects 
on the competence tests.) 

4 . Levin and Baker scrambled the item sequence in a program of question- 
able quality. They suggested that a more effective program would 
probably have been disrupted more by item scrambling than a less 
effective one. Ihe program used in the present study was of good 
quality, but it did not seem to be seriously damaged by item 
scrambling. This may suggest that the teaching ability of a program 
is not what causes criterion test differences in scrambled learning. 
This possibility needs to be further studied. 

5 . There is a possibility that the item scrambling spread out the 
different tasks so that the learning took place by wholes instead 
of by parts, and that the subjects somehow learned the component 
tasks all at the same time. This possibility needs to be further 
explored* 

6. The conclusions suggest that the form of the content being taught 
may be les9 crucial to learning than the programming method by which 
it is taught. Repetition may reduce the harmful effects of 
occasional mis sequencing. It may be that if one cannot construct a 
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logical sequence, then repetitive exposure to the concepts of the 
content may compensate for it. Another tactic may be to construct 
frames which are as discrete as possible so as to avoid undue inter- 
frame dependency. These possibilities need to be studied further. 

7. The results of the study indicated that boys and subjects with 

average IQ's require more time when learning from scrambled sequences. 
Each of these groups represents at least one-half of the population, 

If instructional program builders hope to individualize instruction 
for these two groups, they must see that instructional frames are 
carefully sequenced so as to provide maximum positive transfer from 
one frame to the next , 



A need for more learning time by boys and by learners with average 
IQ's appears to be the most significant result of having nonlogical frame 
sequences in instructional programs. Khen item scrambling does produce 
detrimental effects in learning, these effects are more relative to the 
efficiency of instruction and learning than to the type or amount of 
learning produced by the program, This inefficiency can be a crucial 
matter in learning situations in which time is important. In cases in 
which the time required for learning Is not important, however, the 
sequence in which individual frames and component learning tasks are 
arranged may not be so significant if: 1) the items in the program are 
constructed as discrete learning sets, 2) the concepts being programmed 
are used repetitively in several different situations, and 3) the learner 
is kept quite actively involved while he is advancing through the program. 
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APPENDIX A 
THE COMPETENCE TESTS 



The competence tests measured skills at the comprehension, 
application and analysis levels (Bloom, 1936)* Considerable care was 
given to the construction of the test items such that the instructional 
program did not explicitely "teach the answers" to the competence test 
questions*. For example, the program taught the subjects how to compute 
slopes of lines when two coordinates appear on a line, and when the line 
is on a graph. One type of test item at the analysis level required 
that the subjects compute a slope when the coordinates of two points 
are given, but when no graph is shown and no visible line is given. 

Ihe tests measured whether the subjects could: 

1. Identify the meaning of the terms "parallel" and "nonparallel , " 
(comprehension) 

2. Identify nonparallel lines on a graph in the case in which the 
lines intersect within the graph, (application) 

3. Identify nonparallel lines on a graph in the case in which the 
lines would intersect outside of the graph if they were extended 
far enough, (analysis) 

4. Identify lines fcr which slopes can be computed, (comprehension) 

5. Compute the slope of a line, given the line on a graph and given two 
identified coordinates on the line, (application) 

6. Compute the slope of a line by using two grid intersections through 
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which the line passes. (analysis) 

7. Identify pairs of parallel lines, given a number of lines which 
have their slopes identified. (Comprehension) 

8. Compute the slopes of two lines and identify whether the lines are 
parallel or not. (application) 

9. Compute the slope of a line when given only two sets of coordinate 
pairs through which the line passes, when none of the given in- 
formation appears on a graph, and when the line is not visibly 
indicated. (analysis) 

The KR-20 reliability coefficient of each of the two competence 
tests was at least .84 for the complete test. 

Each competence test was composed of three subtests. Each sub- 
test contained nine test items, all of which measured the same type of 
complexity. The KR-20 reliability coefficients of each of these sub- 
tests is shown in the following table. 



The KR-20 Reliability of the Subtests 
of Complexity 



Subteat of 


Immediate 


Retention 


Complexity 


Posttest 


test 


Comprehension 


.72 


.71 


Application 


.69 


.75 


Analysis 


.73 


.79 



N * 113 subjects taking each subtest 
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APPENDIX B— Continued 
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